1. Field of the Invention
The present invention relates to an image processing method and an image processing apparatus for making a panoramic mosaic image by stitching a plurality of images, a computer program product for realizing the image processing apparatus and a computer memory product storing the computer program.
2. Description of Related Art
To bequeath cultural heritages, arts and craftworks to posterity, work for digitizing images acquired with a camera is ongoing. Instead of recording the digitized images simply as two-dimensional images, in the case of a craftwork of a three-dimensional configuration such as a statue of Buddha or a sculpture for example, recording is performed as a three-dimensional image which is formed by digitizing images acquired in a plurality of directions and performing image-processing with respect to the plural two-dimensional images.
Further, to record a horizontally elongate image which is an object to be acquired such as a Japanese picture scroll as a two-dimensional digital image, the acquisition of images is performed divisionally with a plurality of cameras. Therefore, it is necessary to digitize the divisionally acquired images and seamlessly stitch the images by image processing.
On the other hand, with the increasing use of digital cameras, techniques (image mosaicking) for forming an image having a wide field of view and high resolution are actively studied. For example, a panoramic mosaic image may be formed by calculating geometric transformation such as two-dimensional rigid body transformation or affine transformation between two images and stitching the images utilizing the calculated geometric transformation. Similarly, when three or more images exist, geometric transformation between two images is first calculated to form a panoramic mosaic image, and then geometric transformation between the panoramic mosaic image and an image to be stitched with the panoramic mosaic image is calculated to perform image stitching. In this way, geometric transformation is calculated successively between a panoramic mosaic image and an image to be stitched therewith for performing image stitching.
However, in the case of a Japanese picture scroll, image stitching cannot be accurately performed when the above-described rigid body transformation or affine transformation is simply used. The reason lies in the condition of the acquisition of the images. Since a Japanese picture scroll is very long, it is difficult to acquire the image with the scroll entirely spread, and therefore, the image is acquired divisionally with respect to a number of regions. At that time, when the image which is an object to be acquired is sufficiently far away from the camera or when an image is acquired with a telephoto lens of a very long focal distance, the problem can be simplified because the image thus acquired can be assumed to be an image of parallel projection. Actually, however, acquisition of images cannot help being performed from a limited distance, so that the obtained image generally includes perspective projection distortion.
Moreover, having been formed around the 16th century, Japanese picture scrolls are very old, so that care need be taken in its handling and repetitive acquisition of image is often difficult. Therefore, it is often necessary to use acquired images in the past.
Generally, an image acquired under the above-described conditions cannot be assumed to be an image of parallel projection and includes perspective projection distortion. Therefore, when geometric transformation such as the rigid body transformation or the affine transformation, which does not give careful consideration to perspective projection distortion, is used, a properly stitched image cannot be obtained.
Therefore, planar projective transformation is utilized as a method which gives consideration to perspective projection distortion. FIG. 1 is a schematic diagram illustrating images stitched by planar projective transformation. In performing image stitching by planar projective transformation, similarly to the above, planar projective transformation between two images (image 1 and image 2) is first calculated to form a panoramic mosaic image. Subsequently, planar projective transformation between the panoramic mosaic image (which is the stitched image of image 1 and image 2 in this example) and an image (image 3 in this example) to be stitched with the panoramic mosaic image is calculated to form a panoramic mosaic image. Then, planar projective transformation between the panoramic mosaic image and an image to be stitched therewith is calculated. In this way, image stitching is performed successively.
Although the planar projective transformation can remove perspective projection distortion which cannot be removed by such geometric transformation as the rigid body transformation or the affine transformation, the resulting panoramic mosaic image becomes like one acquired from a certain viewpoint using a wide-angle lens. Specifically, as shown in FIG. 1, the distortion is little in an image 1 located at the point of regard but gradually increases as an image becomes farther away from the image 1.
Further, acquired images often have different scales, and even when such images are stitched together, it is desirable that the picture, or painting in the stitched image has a constant height and that an upper and a lower boundaries of the picture extend substantially parallel with each other from one end to the other end. Since such image stitching for a Japanese picture scroll is difficult by a conventional method, there is a demand for development of an image processing method which is capable of accurately forming a panoramic mosaic image so that the height of the picture becomes constant.